Search for: All records

Writing scientific explanations is a core practice in science. However, students find it difficult to write coherent scientific explanations. Additionally, teachers find it challenging to provide real-time feedback on students’ essays. In this study, we discuss how PyrEval, an NLP technology, was used to automatically assess students’ essays and provide feedback. We found that students explained more key ideas in their essays after the automated assessment and feedback. However, there were issues with the automated assessments as well as students’ understanding of the feedback and revising their essays. 

Writing and revising scientific explanations helps students integrate disparate scientific ideas into a cohesive understanding of science. Natural language processing technologies can help assess students’ writing and give corresponding feedback, which supports their writing and revision of their scientific ideas. However, the feedback is not always helpful to students. Our study investigated 241 middle school students’ a) use of feedback, b) how it affected their revisions, and c) how these factors affected students’ writing improvement. We found that students made more content-related revisions when they used feedback. Making content-related revisions also assisted students in improving their writing. But students still found it difficult to make integrated revisions and did not use feedback often. Additional support to assist students to understand and use feedback, especially for students with limited science knowledge, is needed. 

The Institute for Student‐AI Teaming (iSAT) addresses the foundational question:how to promote deep conceptual learning via rich socio‐collaborative learning experiences for all students?—a question that is ripe for AI‐based facilitation and has the potential to transform classrooms. We advance research in speech, computer vision, human‐agent teaming, computer‐supported collaborative learning, expansive co‐design, and the science of broadening participation to design and study next generation AI technologies (called AI Partners) embedded in student collaborative learning teams in coordination with teachers. Our institute ascribes to theoretical perspectives that aim to create a normative environment of widespread engagement through responsible design of technology, curriculum, and pedagogy in partnership with K–12 educators, racially diverse students, parents, and other community members.

 

To assess student knowledge, educators face a tradeoff between open-ended versus fixed response questions. Open-ended questions are easier to formulate, and provide greater insight into student learning,vbut are burdensome. Machine learning methods that could reduce the assessment burden also have a cost, given that large datasets of reliably assessed examples (labeled data) are required for training and testing. We address the human costs of assessment and data labeling using selective prediction, where the output of a machine learned model is used when the model makes a confident decision, but otherwise the model defers to a human decision-maker. The goal is to defer less often while maintaining human assessment quality on the total output. We refer to the deferral criteria as a deferral policy, and we show it is possible to learn when to defer. We first trained an autograder on a combination of historical data and a small amount of newly labeled data, achieving moderate performance. We then used the autograder output as input to a logistic regression to learn when to defer. The learned logistic regression equation constitutes a deferral policy. Tests of the selective prediction method on a held out test set showed that human-level assessment quality can be achieved with a major reduction of human effort. 

Building causal knowledge is critical to science learning and scientific explanations that require one to understand the how and why of a phenomenon. In the present study, we focused on writing about the how and why of a phenomenon. We used natural language processing (NLP) to provide automated feedback on middle school students’ writing about an underlying principle (the law of conservation of energy) and its related concepts. We report the role of understanding the underlying principle in writing based on NLP-generated feedback. 

Creating effective middle school STEM curricula requires a combination of individual and collaborative learning. Prior studies showed that finding a proper balance and providing uninterrupted knowledge transmission between different learning modes can be challenging in such mixed pedagogical approaches. In this paper, we present a multi-device interactive educational platform named SimSnap to teach biology curriculum to middle school children. SimSnap facilitates interactions among touchscreen Chromebooks to perform in-class individual and group activities. We present a usability analysis study with eight middle school children where they learn about the influence of temperature on tomato plant growth. Our study demonstrated that SimSnap facilitates group discussions to complete collaborative tasks. It also creates seamless knowledge propagation between prior to current tasks to learn about more complex concepts from previous simpler activities. Middle school children gave overall high usability ratings and positive feedback on SimSnap. This study also helped to outline some design recommendations for future improvements of SimSnap. 

Classroom orchestration requires teachers to concurrently manage multiple activities across multiple social levels (individual, group, and class) and under various constraints. Real-time dashboards can support teachers; however, designing actionable dashboards is a huge challenge. This paper describes a participatory design study to identify and inform critical features of a dashboard for displaying relevant, actionable, real-time data. We leveraged a Sense-Assess-Act framework to present dashboard mockups to teachers for feedback. Although the participating teachers differed in how they would use the presented information (during class or after class as a post hoc analysis tool), two common emerging themes were that they wanted to use the data to a) better support their students and b) to make broader instructional decisions. We present data from our study and propose a customizable, mobile dashboard, that can be adapted to a teacher's specific needs at a specific time, to help them better facilitate learning activities. 

Classroom orchestration is a multifaceted pedagogical challenge, requiring teachers to simultaneously manage activities across multiple social levels and under various constraints. Teacher dashboards are commonly developed tools to aid orchestration; however, many fall short in real-time classrooms. To address this impediment, we used participatory design sessions with teachers to better understand their needs, based on which, we plan to build a dynamic dashboard with real-time actionable metrics.